Vehicle collision avoidance apparatus and method

ABSTRACT

A vehicle collision avoidance apparatus includes an interface and a processor. The interface receives a point cloud map representing a surrounding environment and an image of the surrounding environment. The processor is configured to: determine whether an object that is expected to collide with the vehicle is present in the point cloud map; activate an avoidance traveling process; set a collision avoidance space defined to avoid the object according to the avoidance traveling process; identify a type of the object based on a region of interest that is set according to a location of the object in the image; based on identifying the type of the object, determine whether the type of the object corresponds to a set avoidance target; and drive the vehicle to the collision avoidance space in response to a determination that the type of the object corresponds to the set avoidance target.

CROSS-REFERENCE TO RELATED APPLICATION

This present application claims the benefit of priority to Korean PatentApplication No. 10-2019-0135520, filed on Oct. 29, 2019, in the KoreanIntellectual Property Office, the entire disclosure of which isincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a vehicle collision avoidanceapparatus and method for controlling a vehicle such that the vehicle maynot collide with objects around the vehicle.

BACKGROUND

When a vehicle is traveling, a driver's field of view may be limited. Insome cases, even when a camera is installed outside the vehicle, it maybe difficult for the driver to recognize the surrounding environmentquickly and accurately using the camera, which may lead to an accident.

In some examples, as a measure for reducing traffic accidents, camerasmay be installed on the front, rear, left, and right of the outside of avehicle and each camera may be connected to a navigation. Each cameramay photograph a blind spot that the driver cannot see, and provide thephotographed blind spot to the navigation. In some examples, it isdetermined whether a side vehicle is present in an image acquired from acamera of a vehicle, and a steering angle may be controlled to avoid acollision with the side vehicle. In some cases, the driver may bealerted of the presence of the side vehicle so as to prevent a collisionbetween the vehicles from occurring.

In some cases, collision avoidance using cameras may be limited inquickly detecting movement of other vehicles.

SUMMARY

The present disclosure describes a vehicle collision avoidance apparatusthat may quickly and accurately recognize information on an object (forexample, information on at least one of a speed of the object, atraveling direction of the object, a size of the object, or a distancebetween the object and a vehicle) in the surrounding environment using alight detection and ranging device (LIDAR) installed in the vehiclealong with a camera installed in the vehicle.

The present disclosure also describes a vehicle collision avoidanceapparatus that may prevent a collision in advance or minimize damagecaused by a collision by causing the vehicle to perform a collisionprevention operation (for example, deceleration, acceleration, orsteering) in preparation for the possibility of collision between avehicle and a potentially threatening object which can threaten thevehicle, in response to a determination that an object in thesurrounding environment is a potentially threatening object, based on apoint cloud map generated through a LIDAR installed in the vehicle.

The present disclosure may enable a vehicle to avoid a collision with apotentially threatening object by setting a region of interest in animage generated by a camera installed in the vehicle corresponding to aposition of the potentially threatening object present in a point cloudmap generated through a LIDAR installed in the vehicle, determining atype of the potentially threatening object in the region of interest asa set avoidance target, and moving the vehicle to a collision avoidancespace in response to a determination that the vehicle will collide withthe potentially threatening object.

The present disclosure describes a vehicle collision avoidance apparatusthat may prevent a collision between a vehicle and a potentiallythreatening object as effectively as possible by determining a casewhere a collision between the vehicle and the potentially threateningobject is predicted as an emergency situation, and in the emergencysituation temporarily setting as a collision avoidance space not only atravelable area but also an area that in a non-emergency situation is anon-travelable area.

According to one aspect of the subject matter described in thisapplication, a vehicle collision avoidance apparatus includes aninterface and a processor. The interface is configured to: receive, froma light detection and ranging device (LIDAR) installed at a vehicle, apoint cloud map representing a surrounding environment within a setrange from the LIDAR; and receive, from a camera installed at thevehicle, an image of the surrounding environment. The processor isconfigured to: determine whether an object that is expected to collidewith the vehicle is present in the point cloud map; in response to adetermination that the object is present in the point cloud map,activate an avoidance traveling process; set a collision avoidance spacedefined to avoid the object according to the avoidance travelingprocess; identify a type of the object based on a region of interestthat is set according to a location of the object in the image; based onidentifying the type of the object, determine whether the type of theobject corresponds to a set avoidance target; and drive the vehicle tothe collision avoidance space in response to a determination that thetype of the object corresponds to the set avoidance target.

Implementations according to this aspect may include one or more of thefollowing features. For example, the point cloud map may include aplurality of point could maps that are received from the LIDAR based onset intervals, and the processor may be configured to: transform eachpoint cloud map from a three-dimensional point cloud map to atwo-dimensional occupancy grid map (OGM); compare OGMs of the pluralityof point could maps, each OGM including an occupied area in which one ormore objects are expected to be present; based on comparing the OGMs,determine movement of the one or more objects in the occupied area;based on determining the movement of the one or more objects in theoccupied area, determine object information corresponding to the one ormore objects in the occupied area, the object information including atleast one of a speed of the one or more objects, a traveling directionof the one or more objects, a size of the one or more objects, or adistance between the one or more objects and the vehicle; and based onthe object information, determine whether the one or more objectscorrespond to the object that is expected to collide with the vehicle.

In some implementations, the processor may be configured to determinethat the one or more objects correspond to the object that is expectedto collide with the vehicle based on (i) the size of the one or moreobjects being greater than a set size, (ii) the speed of the one or moreobjects toward the vehicle being greater than a set speed, and (iii) thedistance between the one or more objects and the vehicle being less thana set separation distance.

In some implementations, the interface may be configured to receive ahigh definition (HD) map from a server, the HD map including laneinformation, and the processor may be configured to: estimate movementof the one or more objects based on the object information; determinewhether the estimated movement of the one or more objects is normalaccording to the lane information in the HD map; and based on adetermination that the estimated movement of the one or more objects isabnormal, determine that the one or more objects correspond to theobject that is expected to collide with the vehicle.

In some implementations, the interface may be configured to receive aplurality of images from the camera at the set intervals, and each OGMmay further include a non-occupied area in which no object is expectedto be present. The processor may be configured to: estimate movement ofthe one or more objects based on the object information; based on theestimated movement of the one or more objects, determine a travelablearea of the vehicle in the non-occupied area; adjust the travelable areaof the vehicle based on travelable areas determined from the pluralityof images; and control traveling of the vehicle based on the adjustedtravelable area.

In some implementations, the processor may be configured to, beforesetting the collision avoidance space, cause the vehicle to perform acollision prevention operation by decelerating, accelerating, orsteering the vehicle based on a distance between the object and thevehicle being greater than a set braking distance of the vehicle.

In some implementations, the processor may be configured to: determinethat the vehicle is expected to collide with the object based on adistance between the object and the vehicle being less than a setbraking distance of the vehicle; and in response to a determination thatthe vehicle is expected to collide with the object, drive the vehicle tothe collision avoidance space.

In some implementations, the processor may be configured to: set anavoidance area including non-travelable areas that the vehicle is notallowed to enter; set the avoidance area and a travelable area of thevehicle as the collision avoidance space; and drive the vehicle to thecollision avoidance space to avoid a collision between the vehicle andthe object. In some examples, the processor may be configured to, beforedriving the vehicle to the collision avoidance space, transmit a messageabout movement of the vehicle to the collision avoidance space toanother vehicle located within a set range from the vehicle.

In some implementations, the processor may be configured to: in responseto a determination that the object is present in the point cloud map,set an area corresponding to spatial coordinates of the object in theimage as the region of interest; increase a frame rate of the camera forcapturing images of the region of interest; and identify the type of theobject based on the images captured at the frame rate.

In some implementations, the processor may be configured to: perform anobject identification process with the image including the region ofinterest; and based on performance of the object identification process,identify the type of the object and determine whether the type of theobject corresponds to the set avoidance target. For instance, the objectidentification process may include a neural network model trained todetect a sample object in a sample image and identify a type of thesample object.

In some implementations, the processor may be configured to deactivatethe avoidance traveling process in response to (i) a determination thatthe type of the object does not correspond to the set avoidance targetor (ii) a determination that the vehicle is not expected to collide withthe object.

According to another aspect, a vehicle collision avoidance methodincludes: receiving, from a light detection and ranging device (LIDAR)installed at a vehicle, a point cloud map representing a surroundingenvironment within a set range from the LIDAR; receiving, from a camerainstalled at the vehicle, an image of the surrounding environment;determining whether an object that is expected to collide with thevehicle is present in the point cloud map; in response to adetermination that the object is present in the point cloud map,activating an avoidance traveling process; setting a collision avoidancespace defined to avoid the object according to the avoidance travelingprocess; identifying a type of the object based on a region of interestthat is set according to a location of the object in the image; based onidentifying the type of the object, determining whether the type of theobject corresponds to a set avoidance target; and driving the vehicle tothe collision avoidance space in response to a determination that thetype of the object corresponds to the set avoidance target.

Implementations according to this aspect may include one or more of thefollowing features or features similar to those described above for thevehicle collision avoidance apparatus. For example, receiving the pointcloud map may include receiving a plurality of point cloud maps from theLIDAR at set intervals. The method may further include: transformingeach point cloud map from a three-dimensional point cloud map to atwo-dimensional occupancy grid map (OGM); comparing OGMs correspondingto the plurality of point cloud maps, each OGM including an occupiedarea in which one or more objects are expected to be present; based oncomparing the OGMs, determining movement of the one or more objects inthe occupied area; based on determining the movement of the one or moreobjects in the occupied area, determining object informationcorresponding to the one or more objects in the occupied area, theobject information including at least one of a speed of the one or moreobjects, a traveling direction of the one or more objects, a size of theone or more objects, or a distance between the one or more objects andthe vehicle; and based on the object information, determining whetherthe one or more objects correspond to the object that is expected tocollide with the vehicle.

In some examples, determining whether the one or more objects correspondto the object that is expected to collide with the vehicle may include:determining that the one or more objects correspond to the object thatis expected to collide with the vehicle based on (i) the size of the oneor more objects being greater than a set size, (ii) the speed of the oneor more objects toward the vehicle being greater than a set speed, and(iii) the distance between the one or more objects and the vehicle beingless than a set separation distance.

In some examples, the method may further include receiving a highdefinition (HD) map from a server, the HD map including laneinformation. In these or other examples, determining whether the one ormore objects correspond to the object that is expected to collide withthe vehicle may include: estimating movement of the one or more objectsbased on the object information; determining whether the estimatedmovement of the one or more objects is normal according to the laneinformation in the HD map; and based on a determination that theestimated movement of the one or more objects is abnormal, determiningthat the one or more objects correspond to the object that is expectedto collide with the vehicle.

In some implementations, the method may further include, before settingthe collision avoidance space, causing the vehicle to perform acollision prevention operation by decelerating, accelerating, orsteering the vehicle based on a distance between the object and thevehicle being greater than a set braking distance of the vehicle. Insome examples, driving the vehicle to the collision avoidance space mayinclude: determining that the vehicle is expected to collide with theobject based on a distance between the object and the vehicle being lessthan a set braking distance of the vehicle; and in response to adetermination that the vehicle is expected to collide with the object,driving the vehicle to the collision avoidance space.

In some implementations, driving the vehicle to the collision avoidancespace may include: setting an avoidance area including non-travelableareas that the vehicle is not allowed to enter; setting the avoidancearea and a travelable area of the vehicle as the collision avoidancespace; and driving the vehicle to the collision avoidance space to avoida collision between the vehicle and the object.

In some examples, identifying the type of the object in the region ofinterest in the image may include: in response to a determination thatthe object is present in the point cloud map, setting an areacorresponding to spatial coordinates of the object in the image as theregion of interest; increasing a frame rate of the camera for capturingimages of the region of interest; and identifying the type of the objectbased on the images captured at the frame rate.

In addition, a method and a system for implementing the presentdisclosure, and a computer-readable recording medium having a computerprogram stored therein to perform the method, may be further provided.

Other aspects and features as well as those described above will becomeclear from the accompanying drawings, claims, and the detaileddescription of the present disclosure.

In some implementations, it may be possible to quickly and accuratelyrecognize information on an object (for example, information on at leastone of a speed of the object, a traveling direction of the object, asize of the object, or a distance between the object and a vehicle) inthe surrounding environment using a LIDAR installed in the vehicle alongwith a camera installed in the vehicle.

In some implementations, it may be possible to prevent a collision inadvance or minimize damage caused by a collision that does occurs, bycausing a vehicle to perform a collision prevention operation (forexample, deceleration, acceleration, or steering) in preparation for thepossibility of collision between a vehicle and a potentially threateningobject which can threaten the vehicle, in response to a determinationthat an object in the surrounding environment is a potentiallythreatening object, based on a point cloud map generated through a LIDARinstalled in the vehicle.

In some implementations, it may be possible to enable a vehicle to avoida collision with a potentially threatening object by setting a region ofinterest in an image generated by a camera installed in the vehiclecorresponding to a position of the potentially threatening objectpresent in a point cloud map generated through a LIDAR installed in thevehicle, determining a type of the potentially threatening object in theregion of interest as a set avoidance target, and moving the vehicle toa collision avoidance space in response to a determination that thevehicle will collide with the potentially threatening object.

In some implementations, it may be possible to prevent a collisionbetween a vehicle and a potentially threatening object as effectively aspossible by determining a case where a collision between the vehicle andthe potentially threatening object is predicted as an emergencysituation, and in the emergency situation temporarily setting as acollision avoidance space not only a travelable area but also an areathat in a non-emergency situation is a non-travelable area.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of the presentdisclosure will become apparent from the detailed description of thefollowing aspects in conjunction with the accompanying drawings.

FIG. 1 is a diagram illustrating an example vehicle including an examplevehicle collision avoidance apparatus.

FIG. 2 is a block diagram illustrating an example system including thevehicle collision avoidance apparatus.

FIG. 3 is a diagram showing an example of an autonomous vehicle and afifth generation (5G) network in a 5G communication system.

FIG. 4 is a diagram illustrating an example configuration of the vehiclecollision avoidance apparatus.

FIG. 5 is a diagram illustrating another example configuration of thevehicle collision avoidance apparatus.

FIG. 6 is a diagram illustrating an example configuration of determiningmovement and a travelable area of an object using a LIDAR in the vehiclecollision avoidance apparatus.

FIG. 7 is a diagram illustrating an example process for checking apotentially threatening object in the vehicle collision avoidanceapparatus.

FIG. 8 is a diagram illustrating another example process for checkingthe potentially threatening object in the vehicle collision avoidanceapparatus.

FIGS. 9 and 10 are diagrams illustrating an example processing methodfor determining an example of a potentially threatening object in thevehicle collision avoidance apparatus.

FIG. 11 is a flowchart illustrating an example of a vehicle collisionavoidance method.

DETAILED DESCRIPTION

The implementations disclosed in the present specification will bedescribed in greater detail with reference to the accompanying drawings,and throughout the accompanying drawings, the same reference numeralsare used to designate the same or similar components and redundantdescriptions thereof are omitted.

The vehicle described in the present disclosure may include, but is notlimited to, a vehicle having an internal combustion engine as a powersource, a hybrid vehicle having an engine and an electric motor as apower source, and an electric vehicle having an electric motor as apower source.

FIG. 1 is a diagram illustrating an example vehicle including a vehiclecollision avoidance apparatus.

Referring to FIG. 1, in a vehicle 100 to which a vehicle collisionavoidance apparatus is applied, for example, a LIDAR 101 and a camera102 may be installed at the same position. Here, the LIDAR 101 and thecamera 102 may be installed outside the vehicle 100, and may beinstalled at one or more positions (for example, a front surface, a sidesurface, and a rear surface of the vehicle).

The LIDAR 101 may irradiate, for example, vertical and horizontal lightto the surrounding environment in a set range, and generate a pointcloud map of the surrounding environment based on the reflected andreceived light.

In addition, the camera 102 may generate an image of the surroundingenvironment using at least one of a red, green, and blue (RGB) sensor,an infrared radiation (IR) sensor, or a time of flight (TOF) sensor.

The vehicle collision avoidance apparatus is mounted inside the vehicle100, and is able to receive a point cloud map and an image,respectively, from the LIDAR 101 and the camera 102 that photograph theoutside of the vehicle 100, and recognize an object present in thesurrounding environment of the vehicle based on the point cloud map andthe image.

The vehicle collision avoidance apparatus may first activate anavoidance traveling algorithm when an object recognized by the pointcloud map is a potentially threatening object that is likely to threatenthe vehicle 100, perform collision prevention operations (for example,deceleration, acceleration, steering) according to the avoidancetraveling algorithm, and set a collision avoidance space in advance. Inaddition, based on the image received from the camera, the vehiclecollision avoidance apparatus may determine that the type of thepotentially threatening object is a set avoidance target (for example, avehicle or a person), and move the vehicle 100 to the collisionavoidance space set in advance in response to a determination that thevehicle 100 will collide with the potentially threatening object.Accordingly, a collision avoidance response speed of the vehicle 100 canbe increased, and the collision with the potentially threatening objectcan be quickly avoided.

FIG. 2 is a block diagram illustrating an example system including thevehicle collision avoidance apparatus.

Referring to FIG. 2, a system 200 to which the vehicle collisionavoidance apparatus is applied may be included in the vehicle 100, andmay include a transceiver 201, a controller 202, a user interface 203,an object detector 204, a driving controller 205, a vehicle driver 206,an operator 207, a sensor 208, a storage 209, and a vehicle collisionavoidance apparatus 210.

In some implementations, a system to which a vehicle collision avoidanceapparatus is applied may include constituent elements other than theconstituent elements shown and described in FIG. 2, or may not includesome of the constituent elements shown and described in FIG. 2. In someimplementations, the controller 202 may include one or more of thetransceiver 201, the user interface 203, the object detector 204, thedriving controller 205, the storage 209, and the vehicle collisionavoidance apparatus 210.

The vehicle 100 may be switched from an autonomous mode to a manualmode, or switched from the manual mode to the autonomous mode dependingon the driving situation. Here, the driving situation may be judged byat least one of the information received by the transceiver 201, theexternal object information detected by the object detector 204, or thenavigation information acquired by the navigation module.

The vehicle 100 may be switched from the autonomous mode to the manualmode, or from the manual mode to the autonomous mode, according to auser input received through the user interface 203.

When the vehicle 100 is operated in the autonomous driving mode, thevehicle 100 may be operated under the control of the operator 207 thatcontrols driving, parking, and unparking. When the vehicle 100 isoperated in the manual mode, the vehicle 100 may be operated by an inputof the driver's mechanical driving operation.

The transceiver 201 is a module for performing communication with anexternal device. Here, the external device may be a user terminal,another vehicle, or a server.

The transceiver 201 may include at least one of a transmission antenna,a reception antenna, a radio frequency (RF) circuit capable ofimplementing various communication protocols, or an RF element in orderto perform communication.

The transceiver 201 may perform short range communication, GPS signalreception, V2X communication, optical communication, broadcasttransmission/reception, and intelligent transport systems (ITS)communication functions.

The transceiver 201 may further support other functions than thefunctions described, or may not support some of the functions described,in some implementations.

The transceiver 201 may support short-range communication by using atleast one of Bluetooth, Radio Frequency Identification (RFID), InfraredData Association (IrDA), Ultra Wideband (UWB), ZigBee, Near FieldCommunication (NFC), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, orWireless Universal Serial Bus (Wireless USB) technologies.

The transceiver 201 may form short-range wireless communication networksso as to perform short-range communication between the vehicle 100 andat least one external device.

The transceiver 201 may include a Global Positioning System (GPS) moduleor a Differential Global Positioning System (DGPS) module for acquiringposition information of the vehicle 100.

The transceiver 201 may include a module for supporting wirelesscommunication between the vehicle 100 and a server (V2I: vehicle toinfrastructure), between the vehicle 100 and another vehicle (V2V:vehicle to vehicle), or between the vehicle 100 and a pedestrian (V2P:vehicle to pedestrian). That is, the transceiver 201 may include a V2Xcommunication module. The V2X communication module may include an RFcircuit capable of implementing V2I, V2V, and V2P communicationprotocols.

The transceiver 201 may receive a danger information broadcast signaltransmitted by another vehicle through the V2X communication module, andmay transmit a danger information inquiry signal and receive a dangerinformation response signal in response thereto.

The transceiver 201 may include an optical communication module forcommunicating with an external device via light. The opticalcommunication module may include a light transmitting module forconverting an electrical signal into an optical signal and transmittingthe optical signal to the outside, and a light receiving module forconverting the received optical signal into an electrical signal.

The light transmitting module may be formed to be integrated with thelamp included in the vehicle 100.

The transceiver 201 may include a broadcast communication module forreceiving a broadcast signal from an external broadcast managementserver through a broadcast channel, or transmitting a broadcast signalto the broadcast management server. The broadcast channel may include asatellite channel and a terrestrial channel. Examples of the broadcastsignal may include a TV broadcast signal, a radio broadcast signal, anda data broadcast signal.

The transceiver 201 may include an ITS communication module forexchanging information, data, or signals with a traffic system. The ITScommunication module may provide acquired information and data to thetraffic system. The ITS communication module may receive information,data or signals from the traffic system. For example, the ITScommunication module may receive road traffic information from thetraffic system, and provide the information to the controller 202. Forexample, the ITS communication module may receive a control signal fromthe traffic system, and provide the control signal to the controller 202or a processor provided in the vehicle 100.

In some implementations, the overall operation of each module of thetransceiver 201 may be controlled by a separate processor provided inthe transceiver 201. The transceiver 201 may include a plurality ofprocessors, or may not include a processor. When the transceiver 201does not include a processor, the transceiver 201 may be operated underthe control of the processor of another device in the vehicle 100 or thecontroller 202.

The transceiver 201 may implement a vehicle display device together withthe user interface 203. In this case, the vehicle display device may bereferred to as a telematics device or an audio video navigation (AVN)device.

FIG. 3 is a diagram showing an example of operation of an autonomousvehicle and a 5G network in a 5G communication system.

The transceiver 201 may transmit specific information to the 5G networkwhen the vehicle 100 is operated in the autonomous mode (S1).

In this case, the specific information may include autonomousdriving-related information.

The autonomous driving-related information may be information directlyrelated to driving control of the vehicle. For example, the autonomousdriving-related information may include one or more of object dataindicating an object around the vehicle, map data, vehicle state data,vehicle location data, and driving plan data.

The autonomous driving-related information may further include serviceinformation required for autonomous driving. For example, the specificinformation may include information about the destination and thestability level of the vehicle, which are inputted through the userinterface 203.

In addition, the 5G network can determine whether the vehicle isremotely controlled (S2).

Here, the 5G network may include a server or a module which performsremote control related to autonomous driving.

The 5G network may transmit information (or a signal) related to theremote control to an autonomous vehicle (S3).

As described above, the information related to the remote control may bea signal applied directly to the self-driving vehicle, and may furtherinclude service information necessary for autonomous driving. In oneimplementation of the present disclosure, the autonomous vehicle canprovide autonomous driving related services by receiving serviceinformation such as insurance and danger sector information selected ona route through a server connected to the 5G network.

The vehicle 100 is connected to an external server through acommunication network, and is capable of moving along a predeterminedroute without driver intervention using the autonomous drivingtechnology.

In the following implementations, the user may be interpreted as adriver, a passenger, or the owner of a user terminal.

When the vehicle 100 is traveling in the autonomous mode, the type andfrequency of accidents may vary greatly depending on the ability tosense the surrounding risk factors in real time. The route to thedestination may include sectors having different levels of risk due tovarious causes such as weather, terrain characteristics, trafficcongestion, and the like.

At least one of the autonomous vehicle, the user terminal, or the serverof the present disclosure may be linked to or integrated with anartificial intelligence module, a drone (an unmanned aerial vehicle,UAV), a robot, an augmented reality (AR) device, a virtual reality (VR)device, and a device related to 5G services.

For example, the vehicle 100 may operate in association with at leastone AI module or robot included in the vehicle 100, during autonomousdriving.

For example, the vehicle 100 may interact with at least one robot. Therobot may be an autonomous mobile robot (AMR). The mobile robot iscapable of moving by itself, may freely move, and may be equipped with aplurality of sensors so as to be capable of avoiding obstacles duringtraveling. The mobile robot may be a flying robot (for example, a drone)having a flight device. The mobile robot may be a wheeled robot havingat least one wheel and moving by rotation of the wheel. The mobile robotmay be a legged robot having at least one leg and being moved using theleg.

The robot may function as a device that complements the convenience of avehicle user. For example, the robot may perform a function of moving aload placed on the vehicle 100 to the final destination of the user. Forexample, the robot may perform a function of guiding the user, who hasalighted from the vehicle 100, to the final destination. For example,the robot may perform a function of transporting the user, who hasalighted from the vehicle 100, to the final destination.

At least one electronic device included in the vehicle 100 maycommunicate with the robot through a communication device.

At least one electronic device included in the vehicle 100 may providethe robot with data processed by at least one electronic device includedin the vehicle. For example, at least one electronic device included inthe vehicle 100 may provide the robot with at least one of object dataindicating an object around the vehicle, HD map data, vehicle statedata, vehicle position data, or driving plan data.

At least one electronic device included in the vehicle 100 can receivedata processed by the robot from the robot. At least one electronicdevice included in the vehicle 100 can receive at least one of sensingdata, object data, robot state data, robot position data, and movementplan data of the robot, which are generated by the robot.

At least one electronic device included in the vehicle 100 may generatea control signal based on data received from the robot. For example, atleast one electronic device included in the vehicle may compare theinformation about the object generated by the object detection devicewith the information about the object generated by the robot, andgenerate a control signal based on the comparison result. At least oneelectronic device included in the vehicle 100 may generate a controlsignal so as to prevent interference between the route of the vehicleand the route of the robot.

At least one electronic apparatus included in the vehicle 100 mayinclude a software module or a hardware module for implementing anartificial intelligence (AI) (hereinafter referred to as an artificialintelligence module). At least one electronic device included in thevehicle may input the acquired data to the AI module, and use the datawhich is outputted from the AI module.

The artificial intelligence module may perform machine learning on inputdata using at least one artificial neural network (ANN). The artificialintelligence module may output driving plan data through machinelearning on the input data.

At least one electronic device included in the vehicle 100 can generatea control signal based on data which is output from the AI module.

At least one electronic device included in the vehicle 100 may receivedata processed by artificial intelligence, from an external device, viaa communication device, in some implementations. At least one electronicdevice included in the vehicle 1000 may generate a control signal basedon data processed by artificial intelligence.

Artificial intelligence (AI) is an area of computer engineering scienceand information technology that studies methods to make computers mimicintelligent human behaviors such as reasoning, learning, self-improving,and the like.

In addition, artificial intelligence does not exist on its own, but israther directly or indirectly related to a number of other fields incomputer science. In recent years, there have been numerous attempts tointroduce an element of the artificial intelligence into various fieldsof information technology to solve problems in the respective fields.

The controller 202 may be implemented by using at least one of anapplication specific integrated circuit (ASIC), a digital signalprocessor (DSP), a digital signal processing device (DSP), aprogrammable logic device (PLD), a field programmable gate array (FPGA),a processor, a controller, a micro-controller, a microprocessor, orother electronic units for performing other functions.

The user interface 203 is used for communication between the vehicle 100and the vehicle user. The user interface 1300 may receive an inputsignal of the user, transmit the received input signal to the controller202, and provide information held by the vehicle 100 to the user by thecontrol of the controller 202. The user interface 203 may include, butis not limited to, an input module, an internal camera, a bio-sensingmodule, and an output module.

The input module is for receiving information from a user. The datacollected by the input module may be identified by the controller 202and processed by the user's control command.

The input module may receive the destination of the vehicle 100 from theuser and provide the destination to the controller 202.

The input interface may input to the controller 202 a signal fordesignating and deactivating at least one of the plurality of sensormodules of the object detector 204 according to the user's input.

The input module may be disposed inside the vehicle. For example, theinput module may be disposed in one area of a steering wheel, one areaof an instrument panel, one area of a seat, one area of each pillar, onearea of a door, one area of a center console, one area of a head lining,one area of a sun visor, one area of a windshield, or one area of awindow.

The output module is for generating an output related to visual,auditory, or tactile information. The output module may output a soundor an image.

The output module may include at least one of a display module, anacoustic output module, or a haptic output module.

The display module may display graphic objects corresponding to variousinformation.

The display module may include at least one of a liquid crystal display(LCD), a thin film transistor liquid crystal display (TFT LCD), anorganic light emitting diode (OLED), a flexible display, a 3D display,or an e-ink display.

The display module may have a mutual layer structure with a touch inputmodule, or may be integrally formed to implement a touch screen.

The display module may be implemented as a head up display (HUD). Whenthe display module is implemented as an HUD, the display module mayinclude a projection module to output information through an imageprojected onto a windshield or a window.

The display module may include a transparent display. The transparentdisplay may be attached to the windshield or the window.

The transparent display may display a predetermined screen with apredetermined transparency. The transparent display may include at leastone of a transparent thin film electroluminescent (TFEL), a transparentorganic light-emitting diode (OLED), a transparent liquid crystaldisplay (LCD), a transmissive transparent display, or a transparentlight emitting diode (LED). The transparency of the transparent displaymay be adjusted.

The user interface 203 may include a plurality of display modules.

The display module may be disposed on one area of a steering wheel, onearea of an instrument panel, one area of a seat, one area of eachpillar, one area of a door, one area of a center console, one area of ahead lining, or one area of a sun visor, or may be implemented on onearea of a windshield or one area of a window.

The sound output module may convert an electric signal provided from thecontroller 202 into an audio signal, and output the audio signal. Tothis end, the sound output module may include one or more speakers.

The haptic output module may generate a tactile output. For example, thehaptic output module may operate to allow the user to perceive theoutput by vibrating a steering wheel, a seat belt, and a seat.

The object detector 204 is for detecting an object located outside thevehicle 100. The object detector 2400 may generate object informationbased on the sensing data, and transmit the generated object informationto the controller 202. At this time, the object may include variousobjects related to the driving of the vehicle 100, such as a lane,another vehicle, a pedestrian, a motorcycle, a traffic signal, a light,a road, a structure, a speed bump, a landmark, and an animal.

The object detector 204 is a plurality of sensor modules, and mayinclude a camera module as a plurality of image capturers, a lightimaging detection and ranging (LIDAR) device, an ultrasonic sensor, aradio detection and ranging (radar) 1450, and an infrared sensor.

The object detector 204 may sense environmental information around thevehicle 100 through a plurality of sensor modules.

In some implementations, the object detector 204 may further includecomponents other than the components described, or may not include someof the components described.

The radar may include an electromagnetic wave transmitting module and anelectromagnetic wave receiving module. The radar may be implemented by apulse radar system or a continuous wave radar system in terms of theradio wave emission principle. The radar may be implemented using afrequency modulated continuous wave (FMCW) method or a frequency shiftkeying (FSK) method according to a signal waveform in a continuous waveradar method.

The radar may detect an object based on a time-of-flight (TOF) scheme ora phase-shift scheme by using an electromagnetic wave as a medium, andmay detect the position of the detected object, the distance to thedetected object, and a relative speed of the detected object.

The radar may be disposed at an appropriate location outside the vehiclefor sensing an object disposed at the front, back, or side of thevehicle.

The LIDAR may include a laser transmitting module and a laser receivingmodule. The LIDAR may be implemented in a TOF scheme or a phase-shiftscheme.

The LIDAR may be implemented as a driven type or a non-driven type.

When implemented as a driven type, the LIDAR may be rotated by themotor, and is capable of detecting objects around the vehicle 100, andwhen implemented as a non-driven type, the LIDAR may detect objectslocated within a predetermined range on the basis of the vehicle 100.The vehicle 100 may include a plurality of non-driven type LIDARs.

The LIDAR may detect an object based on a TOF scheme or a phase-shiftscheme by using a laser beam as a medium, and may detect the position ofthe detected object, the distance to the detected object, and therelative speed of the detected object.

The LIDAR may be disposed at an appropriate location outside the vehiclefor sensing an object disposed at the front, back, or side of thevehicle.

The image capturer may be disposed at a suitable place outside thevehicle, for example, the front, back, right side mirrors and the leftside mirror of the vehicle, in order to acquire a vehicle exteriorimage. The image acquirer may be a mono camera, but is not limitedthereto, and may be a stereo camera, an around view monitoring (AVM)camera, or a 360 degree camera.

The image capturer may be disposed close to the front windshield in theinterior of the vehicle in order to acquire an image of the front of thevehicle. Alternatively, the image capturer may be disposed around afront bumper or a radiator grill.

The image capturer may be disposed close to the rear glass in theinterior of the vehicle in order to acquire an image of the back of thevehicle. The image capturer may be disposed around the rear bumper, thetrunk, or the tail gate.

The image capturer may be disposed close to at least one side window inthe vehicle in order to obtain an image of the side of the vehicle. Inaddition, the image capturer may be disposed around the fender or door.

The image capturer may provide the acquired image to the controller 202.

The ultrasonic sensor may include an ultrasonic transmission module andan ultrasonic reception module. The ultrasonic sensor can detect anobject based on ultrasonic waves, and can detect the position of thedetected object, the distance to the detected object, and the relativespeed of the detected object.

The ultrasonic sensor may be disposed at an appropriate position outsidethe vehicle for sensing an object at the front, back, or side of thevehicle.

The infrared sensor may include an infrared transmission module and aninfrared reception module. The infrared sensor can detect an objectbased on the infrared light, and can detect the position of the detectedobject, the distance to the detected object, and the relative speed ofthe detected object.

The infrared sensor may be disposed at an appropriate location outsidethe vehicle in order to sense objects located at the front, rear, orside portions of the vehicle.

The controller 202 may control the overall operation of the objectdetector 204.

The controller 202 may compare data sensed by the radar, the LIDAR, theultrasonic sensor, and the infrared sensor with pre-stored data so as todetect or classify an object.

The controller 202 may detect and track objects based on the acquiredimage. The controller 202 may perform operations such as calculating adistance to an object and calculating a relative speed with respect tothe object through an image processing algorithm.

For example, the controller 202 may acquire information on the distanceto the object and information on the relative speed with respect to theobject on the basis of variation of the object size with time in theacquired image.

For example, the controller 202 may obtain information on the distanceto the object and information on the relative speed through, forexample, a pin hole model and road surface profiling.

The controller 202 may detect and track the object based on thereflected electromagnetic wave that is reflected by the object andreturned to the object after being transmitted. The controller 202 mayperform operations such as calculating a distance to an object andcalculating a relative speed of the object based on the electromagneticwave.

The controller 202 may detect and track the object based on thereflected laser beam that is reflected by the object and returned to theobject after being transmitted. The controller 202 may performoperations such as calculating a distance to an object and calculating arelative speed of the object based on the laser beam.

The controller 202 may detect and track the object based on thereflected ultrasonic wave that is reflected by the object and returnedto the object after being transmitted. The controller 202 may performoperations such as calculating a distance to an object and calculating arelative speed of the object based on the ultrasonic wave.

The controller 202 may detect and track the object based on thereflected infrared light that is reflected by the object and returned tothe object after being transmitted. The controller 202 may performoperations such as calculating a distance to an object and calculating arelative speed of the object based on the infrared light.

In some implementations, the object detector 204 may include a separateprocessor from the controller 202. In addition, each of the radar, theLIDAR, the ultrasonic sensor, and the infrared sensor may include aprocessor.

When a processor is included in the object detector 204, the objectdetector 204 may be operated under the control of the processorcontrolled by the controller 202.

The driving controller 205 may receive a user input for driving. In thecase of the manual mode, the vehicle 100 may operate based on the signalprovided by the driving controller 205.

The vehicle driver 206 may electrically control the driving of variousapparatuses in the vehicle 100. The vehicle driver 206 may electricallycontrol driving of a power train, a chassis, a door/window, a safetydevice, a lamp, and an air conditioner in the vehicle 100.

The operator 207 may control various operations of the vehicle 100. Theoperator 207 may be operated in an autonomous mode.

The operator 207 may include a driving module, an unparking module, anda parking module.

In some implementations, the operator 207 may further includeconstituent elements other than the constituent elements to bedescribed, or may not include some of the constitute elements.

The operator 207 may include a processor under the control of thecontroller 202. Each module of the operator 207 may include a processorindividually.

In some implementations, when the operator 207 is implemented assoftware, it may be a sub-concept of the controller 202.

The driving module may perform driving of the vehicle 100.

The driving module may receive object information from the objectdetector 204, and provide a control signal to a vehicle driving moduleto perform the driving of the vehicle 100.

The driving module may receive a signal from an external device via thetransceiver 201, and provide a control signal to the vehicle drivingmodule to perform the driving of the vehicle 100.

The unparking module may perform unparking of the vehicle 100.

The unparking module may receive navigation information from thenavigation module, and provide a control signal to the vehicle drivingmodule to perform the departure of the vehicle 100.

In the unparking module, object information may be received from theobject detector 204, and a control signal may be provided to the vehicledriving module, so that the unparking of the vehicle 100 may beperformed.

The unparking module may receive a signal from an external device viathe transceiver 201, and provide a control signal to the vehicle drivingmodule to perform the unparking of the vehicle 100.

The parking module may perform parking of the vehicle 100.

The parking module may receive navigation information from thenavigation module, and provide a control signal to the vehicle drivingmodule to perform the parking of the vehicle 100.

In the parking module, object information may be provided from theobject detector 204, and a control signal may be provided to the vehicledriving module, so that the parking of the vehicle 100 may be performed.

The parking module may receive a signal from an external device via thetransceiver 201, and provide a control signal to the vehicle drivingmodule so as to perform the parking of the vehicle 100.

The navigation module may provide the navigation information to thecontroller 202. The navigation information may include at least one ofmap information, set destination information, route informationaccording to destination setting, information about various objects onthe route, lane information, or current location information of thevehicle.

The navigation module may provide the controller 202 with a parking lotmap of the parking lot entered by the vehicle 100. When the vehicle 100enters the parking lot, the controller 202 receives the parking lot mapfrom the navigation module, and projects the calculated route and fixedidentification information on the provided parking lot map so as togenerate the map data.

The navigation module may include a memory. The memory may storenavigation information. The navigation information may be updated by theinformation received through the transceiver 201. The navigation modulemay be controlled by a built-in processor, or may be operated byreceiving an external signal, for example, a control signal from thecontroller 202, but the present disclosure is not limited to thisexample.

The driving module of the operator 207 may be provided with thenavigation information from the navigation module, and may provide acontrol signal to the vehicle driving module so that driving of thevehicle 100 may be performed.

The sensor 208 may sense the state of the vehicle 100 using a sensormounted on the vehicle 100, that is, a signal related to the state ofthe vehicle 100, and obtain movement route information of the vehicle100 according to the sensed signal. The sensor 208 may provide theobtained movement route information to the controller 202.

The sensor 208 may include a posture sensor (for example, a yaw sensor,a roll sensor, and a pitch sensor), a collision sensor, a wheel sensor,a speed sensor, a tilt sensor, a weight sensor, a heading sensor, a gyrosensor, a position module, a vehicle forward/reverse movement sensor, abattery sensor, a fuel sensor, a tire sensor, a steering sensor byrotation of a steering wheel, a vehicle interior temperature sensor, avehicle interior humidity sensor, an ultrasonic sensor, an illuminancesensor, an accelerator pedal position sensor, and a brake pedal positionsensor, but is not limited thereto.

The sensor 208 may acquire sensing signals for information such asvehicle posture information, vehicle collision information, vehicledirection information, vehicle position information (GPS information),vehicle angle information, vehicle speed information, vehicleacceleration information, vehicle tilt information, vehicleforward/reverse movement information, battery information, fuelinformation, tire information, vehicle lamp information, vehicleinterior temperature information, vehicle interior humidity information,a steering wheel rotation angle, vehicle exterior illuminance, pressureon an acceleration pedal, and pressure on a brake pedal.

The sensor 208 may further include an acceleration pedal sensor, apressure sensor, an engine speed sensor, an air flow sensor (AFS), anair temperature sensor (ATS), a water temperature sensor (WTS), athrottle position sensor (TPS), a TDC sensor, a crank angle sensor(CAS).

The sensor 208 may generate vehicle state information based on sensingdata. The vehicle status information may be information generated basedon data sensed by various sensors provided in the vehicle.

Vehicle state information may include information such as attitudeinformation of the vehicle, speed information of the vehicle, tiltinformation of the vehicle, weight information of the vehicle, directioninformation of the vehicle, battery information of the vehicle, fuelinformation of the vehicle, tire air pressure information of thevehicle, steering information of the vehicle, interior temperatureinformation of the vehicle, interior humidity information of thevehicle, pedal position information, and vehicle engine temperatureinformation.

The storage 209 is electrically connected to the controller 202. Thestorage 209 may store basic data for each unit of the vehicle collisionavoidance apparatus 210, control data for operation control of each unitof the vehicle collision avoidance apparatus 210, and input/output data.The storage 209 may be various storage devices such as a ROM, a RAM, anEPROM, a flash drive, and a hard drive, in terms of hardware. Thestorage 209 may store various data for overall operation of the vehicle100, such as a program for processing or controlling the controller 202,in particular driver propensity information. Here, the storage 209 maybe formed integrally with the controller 202 or may be implemented as asub-component of the controller 202.

The vehicle collision avoidance apparatus 210 may quickly recognize anobject in the surrounding environment of the vehicle 100 by using aLIDAR and a camera installed in the vehicle 100. The vehicle collisionavoidance apparatus 210 may determine that an object is a potentiallythreatening object and is a set avoidance target, and may control thevehicle 100 to avoid collision with the potentially threatening objectin response to a determination that the vehicle 100 will collide withthe potentially threatening object.

The vehicle collision avoidance apparatus 210 may include an interface,a processor, and a memory, which will be described in detail below withreference to FIG. 4. For example, the interface may be included in thetransceiver 201, the processor may be included in the controller 202,and the memory may be included in the storage 209.

FIG. 4 is a diagram illustrating an example of a configuration of thevehicle collision avoidance apparatus.

Referring to FIG. 4, a vehicle collision avoidance apparatus 400 mayinclude an interface 401, a processor 402, and a memory 403.

The interface 401 may receive a point cloud map of the surroundingenvironment in a set range from a LIDAR installed in a vehicle at setintervals. The interface 401 may receive an image of the surroundingenvironment from a camera installed in the vehicle at the set intervals.Here, the surrounding environment may be the environment of a travelingdirection of the vehicle, but is not limited thereto, and may be theenvironment of all directions with respect to the vehicle.

In addition, the interface 401 may receive a high definition map (HDMap) from a server. Here, the HD Map may be a detailed map of an areaincluding the surrounding environment of the set range or an area (forexample, a district or a neighborhood) in which the vehicle is located.

Upon recognition of an object in the point cloud map, the processor 402may determine whether the object is a potentially threatening objectwhich can threaten the vehicle, thus determining whether a potentiallythreatening object is present in the point cloud map.

In response to determining that a potentially threatening object that islikely to threaten the vehicle is present in the point cloud map, theprocessor 402 may (i) activate an avoidance traveling algorithm to allowthe vehicle to improve the collision avoidance response speed inpreparation of the possibility of collision of the vehicle with thepotentially threatening object, thereby preventing the collision inadvance, or minimize damage caused by a collision that does occur. Inthis case, the processor 402 may, for example, cause the vehicle toperform a collision prevention operation by decelerating, accelerating,or steering the vehicle.

Thereafter, the processor 402 may (ii) set a collision avoidance spacefor avoiding the potentially threatening objects according to theactivated avoidance traveling algorithm, and (iii) identify a type ofthe potentially threatening object in a region of interest (ROI) in theimage corresponding to the position of the potentially threateningobject, and determine that the type of the potentially threateningobject is a set avoidance target (for example, a vehicle or a person).The processor 402 may move the vehicle to the collision avoidance spaceset in advance in response to a determination that the type of thepotentially threatening object is a set avoidance target.

In addition, the processor 402 may further determine whether the vehiclewill collide with the potentially threatening object, and in response toa determination that the vehicle will collide with the potentiallythreatening object, the processor 402 may move the vehicle to thecollision avoidance space, thereby allowing the vehicle to avoid thecollision with the potentially threatening object.

As a result, the processor 402 initially performs the collisionprevention operation by activating the avoidance traveling algorithmbased on the point cloud map received from the LIDAR, and thensecondarily moves the vehicle to the collision avoidance space set inadvance based on the image received from the camera, thereby allowingthe vehicle to quickly avoid the collision with the potentiallythreatening object.

Upon determining the presence of the potentially threatening object, theprocessor 402 may first transform each of the point cloud maps receivedfrom the LIDAR at the set intervals from a three-dimensional point cloudmap to a two-dimensional occupancy grid map (OGM). In this case, theprocessor 402 removes unnecessary parts (for example, the road surfaceand noise) from the three-dimensional point cloud map so as to reducethe amount of data for later object recognition and space recognition,thereby saving resources used for computation. In the point cloud mapfrom which the unnecessary parts have been removed, the processor 402may classify an area as an occupied area in which an object is estimatedto be present or a non-occupied area in which no object is estimated tobe present. Thereafter, the processor 402 may transform thethree-dimensional point cloud map to the two-dimensional occupancy gridmap based on the occupied and non-occupied areas.

Therefore, the two-dimensional occupancy grid map may also include theoccupied area in which an object is estimated to be present and thenon-occupied area in which no object is estimated to be present.

The processor 402 may check the movement of the occupied area in whichan object is estimated to be present in each occupancy grid map, as aresult of comparing each of the transformed occupancy grid maps (aplurality of occupancy grid maps). The processor 402 may checkinformation on an object corresponding to the occupied area based on themovement, and determine whether the object is a potentially threateningobject based on the information on the object. Here, the information onthe object may include at least one of a speed of the object, atraveling direction of the object, a size of the object, or a distancebetween the object and the vehicle. In this case, the processor 402 maycheck the information on the object using, for example, a Kalman filterin the occupied area.

In detail, the processor 402 may compare a first occupancy grid maptransformed from a first point cloud map that was received a set period(for example, 0.1 seconds) before the current time point, with a secondoccupancy grid map transformed from a second point cloud map received atthe current time point. As a result of the comparison, the processor 402may then determine whether the object estimated to be present in theoccupied area is a potentially threatening object depending on whetherthe movement (or size or distance) of the occupied area in which theobject is estimated to be present satisfies a set condition (forexample, movement speed, direction, and size). That is, the processor402 may determine that the object estimated to be present in theoccupied area is a potentially threatening object in response to themovement (or size or distance) of the occupied area in which the objectis estimated to be present satisfying the set condition.

For example, in response to an object estimated to be present in a firstoccupied area in the first occupancy grid map moving to a secondoccupied area in the second occupancy grid map, the processor 402 maydetermine that the object is a potentially threatening object based onthe object being larger than a set size and moving toward the vehicleincluding the vehicle collision avoidance apparatus at above a setspeed, and on the distance between the object and the vehicle beingshorter than a set separation distance. In this case, the processor 402may calculate the speed of the object based on the period in which thethree-dimensional point cloud map is received and the moving distancebetween the first and second occupied areas.

In some implementations, the processor 402 may estimate (or predict) themovement of the object based on the information on the object. Here, theprocessor 402 may check whether the movement of the object is normalbased on the lane information in the HD Map received from the serverthrough the interface 401, and re-determine whether the object is apotentially threatening object based on the checked result. In thiscase, the processor 402 may transform the HD Map into a two-dimensionaloccupancy grid map, check whether the movement of the object is normalbased on lane information within the occupancy grid map transformed fromthe HD Map, and in response to the movement of the object not beingnormal as a result of the checking, determining that the object is apotentially threatening object. For example, when the object estimatedto be present in the occupied area disregards the lane information andquickly approaches the position of the vehicle including the vehiclecollision avoidance apparatus, the processor 402 determines that themovement of the object is abnormal, and thereby determines that theobject is a threatening object.

By contrast, in response to the movement of the object being normal as aresult of checking, the processor 402 may determine that the object isnot a potentially threatening object. For example, even when an objectestimated to be present in the occupied area quickly approaches theposition of the vehicle including the vehicle collision avoidanceapparatus, in response to a prediction that the object is entering anadjacent lane based on the lane information, the processor 402 maydetermine that the movement of the object is normal, and therebydetermine that the object is a non-threatening object that is unlikelyto threaten the vehicle.

When the collision prevention operation of the vehicle is performed, theprocessor 402 may further check the distance between the potentiallythreatening object and the vehicle, in addition to the presence of thepotentially threatening object in the point cloud map, and as thechecked result, may control the vehicle to perform the collisionprevention operation based on the distance being longer than a setbraking distance of the vehicle.

In addition, when determining whether the type of the potentiallythreatening object is an avoidance target, the processor 402 may set anarea corresponding to the spatial coordinates of the potentiallythreatening object in the image received from the camera through theinterface 401 as a region of interest in response to a determinationthat the potentially threatening object is present in the point cloudmap generated by the LIDAR, and identify the type of the potentiallythreatening object present in the set region of interest. In this case,the processor 402 may identify the type of the potentially threateningobject by setting the region of interest in an image receivedsimultaneously with the point cloud map in which it has been determinedthat the potentially threatening object is present (for example, animage received at the same period as the point cloud map, or an imagereceived later than the point cloud map by a set period), and may thenapply an object identification algorithm stored in the memory 403 to theimage of the region of interest. Here, the object identificationalgorithm may be a neural network model trained to detect an object in adesignated area in a collected image and identify the type of thedetected object.

That is, the processor 402 may set the position of the object determinedto be the potentially threatening object in the point cloud mapgenerated by the LIDAR as the region of interest in the image generatedby the camera, and thereby quickly recognize the object within theregion of interest. Accordingly, the processor 402 can quickly andeasily recognize the information on the potentially threatening object(for example, the position, type, size, and form of the potentiallythreatening object), using the LIDAR that is capable of quickly andaccurately estimating the position of the object and the camera that iscapable of quickly and accurately estimating the type (or form or size)of the object.

Furthermore, in response to a determination that the potentiallythreatening object is present in the point cloud map (or in response tothe region of interest being set in the image), the processor 402 mayincrease a frame rate by a set multiple (or a set numerical value) whenthe camera photographs the region of interest, and by the increasedamount of the frame rate may increase the number of times of identifyingthe type of the potentially threatening object in the region of interestwithin each image received through the interface 401. Accordingly, theaccuracy of recognizing the type of potentially threatening object canbe increased to above a set reliability. For example, in a state inwhich the point cloud map and the image are each received every 0.1seconds, in response to a determination that the potentially threateningobject is present in the point cloud map, the processor 402 may increasethe frame rate when the camera photographs the region of interest suchthat an image is received every 0.05 seconds. Accordingly, thepotentially threatening object in the region of interest is identifiedin two images during the same time (0.1 seconds), thus increasing thenumber of times of identifying the type of the potentially threateningobject twofold.

In addition, in response to a determination that the potentiallythreatening object is present in the point cloud map (or in response tothe region of interest being set in the image), the processor 402 mayreceive a larger amount of data (point cloud maps and images) byreducing the period at which the point cloud map generated by the LIDARand the image generated by the camera are generated, thereby providingan environment in which the change in the surrounding environment may beunderstood more quickly.

When determining whether the vehicle and the potentially threateningobject will collide, the processor 402 may check the distance betweenthe potentially threatening object and the vehicle, and, as a result ofthe checking, determine that the vehicle will collide with thepotentially threatening object based on the distance between thepotentially threatening object and the vehicle being shorter than theset braking distance of the vehicle, and move the vehicle to thecollision avoidance space.

In addition, the processor 402 may determine a travelable area of thevehicle from the point cloud map and the image. In this case, theprocessor 402 may determine the travelable area of the vehicle based ona non-occupied area in which it is estimated from the two-dimensionaloccupancy grid map transformed from the three-dimensional point cloudmap that no object is present, and on the movement of the object. Theprocessor 402 may adjust the travelable area of the determined vehiclebased on the travelable area in the image, and control the traveling ofthe vehicle based on the adjusted travelable area. In this case, theprocessor 402 may recognize, for example, travelable areas of thevehicle excluding the object from the point cloud map and the image,respectively, and among the respective recognized travelable areas ofthe vehicle, the processor 402 may determine only an area for which thepoint cloud map and the image match each other as the travelable area ofthe vehicle.

When setting the collision avoidance space, the processor 402 may set,among the non-travelable areas of the vehicle, an avoidance areaaccording to a set condition (for example, a crosswalk or a sidewalkwhere no person is present), and may set the set avoidance area and thetravelable area of the vehicle as the collision avoidance space. Inother words, the processor 402 may determine a situation in which acollision between the vehicle and the potentially threatening object ispredicted as an emergency situation, and in the emergency situation maytemporarily set as a collision avoidance space not only a travelablearea but also an area that in a non-emergency situation is anon-travelable area. In this case, when performing the collisionprevention operation in the vehicle, the processor 402 may, for example,set the collision avoidance space in advance or set the collisionavoidance space every set period, thereby quickly moving the vehicle theset collision avoidance space upon determination that the vehicle willcollide with the potentially threatening object.

When controlling the vehicle to avoid the collision between the vehicleand the potentially threatening object, the processor 402 may move thevehicle to the set collision avoidance space, but before the moving thevehicle to the set collision avoidance space may transmit a message onthe movement of the vehicle to the collision avoidance space to anothervehicle located within a set range with respect to the vehicle, therebyallowing the other vehicle to recognize the moving position of thevehicle in advance so as to prevent collision between the vehicle andthe other vehicle.

Furthermore, the processor 402 may deactivate the avoidance travelingalgorithm in response to a determination that the type of thepotentially threatening object is not the set avoidance target or thatthe vehicle will not collide with the potentially threatening object,thereby reducing energy consumed by the avoidance traveling algorithm.

The memory 403 may store an object identification algorithm, which is aneural network model trained to detect objects in a designated area inan image collected in advance and identify a type of the detectedobject.

The memory 403 may perform a function of temporarily or permanentlystoring data processed by the processor 402. Here, the memory 403 mayinclude a magnetic storage medium or a flash storage medium, but thescope of the present disclosure is not limited thereto. The memory 403may include an internal memory and/or an external memory and may includea volatile memory such as a DRAM, a SRAM or a SDRAM, and a non-volatilememory such as one-time programmable ROM (OTPROM), a PROM, an EPROM, anEEPROM, a mask ROM, a flash ROM, a NAND flash memory or a NOR flashmemory, a flash drive such as an SSD, a compact flash (CF) card, an SDcard, a Micro-SD card, a Mini-SD card, an XD card or memory stick, or astorage device such as a HDD.

FIG. 5 is a diagram illustrating another example of the configuration ofthe vehicle collision avoidance apparatus.

Referring to FIG. 5, a vehicle collision avoidance apparatus 500 may beincluded in a vehicle, and may include a first recognizer 501, a firstestimator 502, a second recognizer 503, a second estimator 504, anobject movement determiner 505, a travelable area determiner 506, acollision prevention operator 507, a collision avoidance area setter508, a collision prediction determiner 509, and a collision avoider 510.Here, each component in the vehicle collision avoidance apparatus 500may correspond to the processor of FIG. 4. In some examples, a processoror controller (e.g., controller 202) may include one or more of thecomponents in FIG. 5. In some examples, the components in FIG. 5 may besoftware modules configured to be executed by the processor orcontroller.

The first recognizer 501 may receive a point cloud map of thesurrounding environment from a LIDAR installed in the vehicle at the setintervals. Upon receiving the point cloud map, the first recognizer 501may recognize, from the point cloud map, at least one of information onan object, spatial information, or lane information. The firstrecognizer 501 may include a first object recognizer 501-1, a firstspace recognizer 501-2, and a first lane recognizer 501-3. Here, thefirst object recognizer 501-1 may recognize the object from the pointcloud map, and recognize information on the object (for example,information on at least one of a speed of the object, a travelingdirection of the object, a size of the object, or a distance between theobject and the vehicle) from the plurality of point cloud maps receivedat the set intervals. The first space recognizer 501-2 may recognize thespatial information from the point cloud map. In addition, the firstlane recognizer 501-3 may recognize the lane information from the pointcloud map.

The first estimator 502 may receive, from the first recognizer 501, atleast one of the information on the object, the spatial information, orthe lane information, and estimate at least one of the movement or thetravelable area of the object based on the received information. Thefirst estimator 502 may include a first object movement estimator 502-1and a first travelable area estimator 502-2. Here, the first objectmovement estimator 502-1 may receive the information on the object fromthe first object recognizer 501-1, and estimate the movement of theobject based on the received information on the object. In this case,the first object movement estimator 502-1 may determine whether theobject recognized from the point cloud map generated by the LIDAR is apotentially threatening object that is likely to threaten the vehiclebased on the movement of the object. For example, the first objectmovement estimator 502-1 may determine that the object is a potentiallythreatening object based on the object being larger than a set size andmoving toward the vehicle at above a set speed, and on the distancebetween the object and the vehicle being shorter than a set separationdistance.

In some implementations, in response to a determination that the objectis a potentially threatening object, the first object movement estimator502-1 may provide an environment in which the type of the object can bequickly identified by transmitting the spatial coordinates of thepotentially threatening object to the second object recognizer 503-1.

The first travelable area estimator 502-2 may receive the spatialinformation from the first space recognizer 501-2, receive the laneinformation from the first lane recognizer 501-3, and estimate thetravelable area of the vehicle based on the received spatial informationand lane information. In addition, the first travelable area estimator502-2 may further receive the information on the object (or the movementof the object) from the first object movement estimator 502-1, and mayestimate the travelable area of the vehicle further based on theinformation on the object (or the movement of the object) along with thespatial information and the lane information.

The second recognizer 503 may receive an image of the surroundingenvironment from a camera installed in the vehicle at the set intervals.Upon receiving the image, the second recognizer 503 may recognize atleast one of the information on the object, the spatial information, orthe lane information from the image. The second recognizer 503 mayinclude a second object recognizer 503-1, a second space recognizer503-2, and a third lane recognizer 503-3. Here, the second objectrecognizer 503-1 may recognize the object from the image, and recognizethe information on the object (for example, information on at least oneof the speed of the object, the traveling direction of the object, thesize of the object, or the distance between the object and the vehicle)from the plurality of images received at the set intervals.

In some implementations, upon receiving the spatial coordinates for thepotentially threatening object from the first object movement estimator502-1, the second object recognizer 503-1 may set an area correspondingto the spatial coordinates in the image as a region of interest (ROI),and may quickly identify the type of the potentially threatening objectin the set region of interest and provide the identified type of thepotentially threatening object to the collision prediction determiner509, such that the collision prediction determiner 509 may quicklydetermine whether the object will collide with the vehicle.

The second space recognizer 503-2 may recognize the spatial informationfrom the image. In addition, the second lane recognizer 503-3 mayrecognize the lane information from the image.

The second estimator 504 may receive at least one of the information onthe object, the spatial information, or the lane information from thesecond recognizer 503, and estimate at least one of the movement or thetravelable area of the object based on the received information. Thesecond estimator 504 may include a second object movement estimator504-1 and a second travelable area estimator 504-2. Here, the secondobject movement estimator 504-1 may receive the information on theobject from the second object recognizer 503-1, and estimate themovement of the object based on the received information on the object.The second travelable area estimator 504-2 may receive the spatialinformation from the second space recognizer 503-2, receive the laneinformation from the second lane recognizer 503-3, and estimate thetravelable area of the vehicle based on the received spatial informationand lane information.

The object movement determiner 505 may receive the movement of theobject estimated from the first object movement estimator 502-1, receivethe movement of the object estimated from the second object movementestimator 504-1, and determine the movement of the object based on eachof the received movements of the object. In this case, for example, theobject movement determiner 505 may determine, as the movement of theobject, only a movement for which the respective estimated movements ofthe object match each other, and predict the future movement of theobject (for example, the direction in which the object intends to move,or the speed of the object) based on the determined movement of theobject.

The travelable area determiner 506 may receive the estimated travelablearea of the vehicle from the first travelable area estimator 502-2,receive the estimated travelable area of the vehicle from the secondtravelable area estimator 504-2, and determine the travelable area ofthe vehicle based on each of the received travelable areas of thevehicle. In this case, for example, the travelable area determiner 506may determine, as the travelable area of the vehicle, only an area forwhich the respective estimated travelable areas of the vehicle matcheach other.

The collision prevention operator 507 may activate the avoidancetraveling algorithm in response to the first object movement estimator502-1 identifying the object recognized from the point cloud mapgenerated by the LIDAR as the potentially threatening object, andaccordingly, may cause the vehicle to perform the collision preventionoperation in preparation of the possibility of collision of the vehiclewith the potentially threatening object. In this case, the collisionprevention operator 507 may cause the vehicle to perform the collisionprevention operation by decelerating, accelerating, or steering thevehicle.

The collision avoidance area setter 508 may set, among thenon-travelable areas of the vehicle, an avoidance area according to aset condition (for example, a crosswalk or a sidewalk where no person ispresent), and may set the set avoidance area as the collision avoidancespace along with the travelable area of the vehicle determined by thetravelable area determiner 506.

The collision prediction determiner 509 may determine that the vehiclewill collide with the potentially threatening object based on the secondobject recognizer 503-1 identifying the type of the potentiallythreatening object detected from the image generated by the camera anddetermining that the type of the potentially threatening object is theset avoidance target, and determining that the distance between thepotentially threatening object and the vehicle is shorter than the setbraking distance of the vehicle.

In response to a determination by the collision prediction determiner509 that the vehicle will collide with the potentially threateningobject, the collision avoider 510 may move the vehicle performing thecollision prevention operation to the collision avoidance space that isset in advance by the collision avoidance area setter 508, therebyquickly moving the vehicle to a safe place at which collision with thepotentially threatening object is unlikely.

FIG. 6 is a diagram illustrating an example of a configuration ofdetermining a movement and a travelable area of an object using a LIDARin the vehicle collision avoidance apparatus.

Referring to FIG. 6, the vehicle collision avoidance apparatus 600includes a road surface remover 601, a recognizer 602, an estimator 603,an object movement determiner 604, and a travelable area determiner 606.

The road surface remover 601 may receive a point cloud map of thesurrounding environment from a LIDAR installed in a vehicle at the setintervals. At this time, the road surface remover 601 may removeunnecessary portions (for example, the road surface (road) and noise)from the point cloud map, thereby saving resources used for computationby reducing the amount of data for later object recognition and spacerecognition.

The road surface remover 601 may use, for example, an algorithm such asan elevation map, and may measure a height for each cell in a bird's eyeview space, and may regard an area in which the measured valuescontinuously change as a road surface and regard an area in which themeasured values discontinuously change as an object area.

The recognizer 602 may receive the point cloud map from which the roadsurface (road) and noise have been removed, and estimate the informationon the object and the spatial information from the received point cloudmap. The recognizer 602 may include an object recognizer 602-1 and aspace recognizer 602-2.

The object recognizer 602-1 may recognize the object from the pointcloud map from which the road surface (road) and the noise have beenremoved, and recognize the information on the object (for example,information on at least one of a probability of the object, a speed ofthe object, a traveling direction of the object, a size of the object,or a distance between the object and the vehicle) from the plurality ofpoint cloud maps received at the set intervals. In this case, the objectrecognizer 602-1 may perform three-dimensional distance-based clusteringon the point cloud map to recognize the information on the object. Theobject recognizer 602-1 may recognize the object using a deep neuralnetwork (DNN) algorithm that is pre-trained to recognize an object froma point cloud map.

From the point cloud map from which the road surface (road) and thenoise have been removed, the space recognizer 602-2 may calculate aprobability of the bird's eye view-based space and an empty space inaddition to the bird's eye view-based space. In this case, the spacerecognizer 602-2 may use, for example, an occupancy grid map (OGM) forspace recognition, and by using data in the height direction mayrecognize the space as an occupied area when the height is higher than apredetermined level and may recognize the space as a non-occupied areawhen the height is less than the predetermined level.

The estimator 603 may include an object movement estimator 603-1 and atravelable area estimator 603-2.

The object movement estimator 603-1 may receive the information on theobject from the object recognizer 602-1, and estimate the movement ofthe object based on the received information on the object. In thiscase, the object movement estimator 603-1 may estimate a predicted pathof the object based on the estimated object movement. Here, the objectmovement estimator 603-1 may estimate future data based on past andpresent data using, for example, a Kalman filter.

The travelable area estimator 603-2 may estimate the travelable area ofthe vehicle based on the bird's eye view-based occupied area receivedfrom the space recognizer 602-2 and on the information on the object (orthe movement of the object) received from the object movement estimator603-1.

The object movement determiner 604 may receive the estimated movement ofthe object estimated from the object movement estimator 603-1 or thepredicted path of the estimated object, and receive camera-basedinformation 605 on an object from the object movement estimatorassociated with the camera. In this case, the object movement determiner604 may determine information on one object by fusing LIDAR-basedinformation on the object and camera-based information on the object.Here, the object movement determiner 604 may further improve theaccuracy of the position of the object and the distance between thevehicle and the object by assigning a weight to the position of theobject and the distance between the vehicle and the object among theinformation on the object acquired through the LIDAR, and furtherimprove the accuracy of recognizing the type, form, and size of theobject by assigning a weight to the type, form, and size of the objectamong the information on the object acquired through the camera.Accordingly, the advantages of both the LIDAR and the camera can beused.

The travelable area determiner 606 may receive an estimated travelablearea 607 of the vehicle from the travelable area estimator associatedwith the camera, and receive the camera-based travelable area from thetravelable area estimator associated with the camera. In this case, thetravelable area determiner 606 may fuse the LIDAR-based travelable areaof the vehicle and the camera-based travelable area of the vehicle so asto determine the travelable area of one vehicle. Here, the travelablearea determiner 606 may assign a weight to the position of thetravelable area of the vehicle and the distance between the vehicle andthe travelable area that are acquired through the LIDAR, thereby moreaccurately acquiring the position of the travelable area of the vehicleand the distance between the vehicle and the travelable area.

FIG. 7 is a diagram illustrating an example of identifying a potentiallythreatening object in the vehicle collision avoidance apparatus.

Referring to FIG. 7, the vehicle collision avoidance apparatus in avehicle may transform each of the point cloud maps received from theLIDAR at set intervals from a three-dimensional point cloud map to atwo-dimensional occupancy grid map (OGM). Here, the two-dimensionaloccupancy grid map may include an occupied area in which an object isestimated to be present and a non-occupied area in which no object isestimated to be present.

In this case, the vehicle collision avoidance apparatus may check theinformation on the object corresponding to the occupied area includingat least one of the speed of the object, the traveling direction of theobject, the size of the object, or the distance between the object andthe vehicle, based on movement of the occupied area in which the objectis estimated to be present in each of the transformed occupancy gridmaps, and determine that the object is a potentially threatening objectbased on the movement of the object according to the information on theobject. For example, the vehicle collision avoidance apparatus maycompare a first occupancy grid map 701 transformed from a first pointcloud map that was received a set period (for example, 0.1 seconds)before the current time point, with a second occupancy grid map 702transformed from a second point cloud map received at the current timepoint. As a result of the comparison, the vehicle collision avoidanceapparatus may determine whether the object estimated to be present inthe occupied area 704 is a potentially threatening object depending onwhether the movement (703→704) of the occupied area in which the objectis estimated to be present satisfies a set condition (for example,movement speed, direction, and size). That is, the vehicle collisionavoidance apparatus may determine that the object estimated to bepresent in the occupied area 704 is a potentially threatening object inresponse to the movement of the occupied area in which the object isestimated to be present satisfying the set condition.

For example, in response to the object estimated to be present in thefirst occupied area 703 moving to the second occupied area 704, thevehicle collision avoidance apparatus may determine that the object is apotentially threatening object based on the object moving toward avehicle 705 including the vehicle collision avoidance apparatus at abovea set speed, and on the distance between the object and the vehicle 705being shorter than a set separation distance.

FIG. 8 is a diagram illustrating another example of checking thepotentially threatening object in the vehicle collision avoidanceapparatus.

Referring to FIG. 8, the vehicle collision avoidance apparatus in thevehicle may compare, for example, a first occupancy grid map transformedfrom a first point cloud map that was received a set period (forexample, 0.1 seconds) before the current time point, with a secondoccupancy grid map transformed from a second point cloud map received atthe current time point, and check whether the object is a potentiallythreatening object depending on whether the movement of the firstoccupancy grid map and the second occupancy grid map 801 of an occupiedarea 802 in which the object is estimated to be present satisfies a setcondition (for example, movement speed, direction, and size). That is,the vehicle collision avoidance apparatus may determine that the objectestimated to be present in the occupied area 802 is a potentiallythreatening object in response to the movement of the occupied area 802in which the object is estimated to be present satisfying the setcondition.

In some implementations, the vehicle collision avoidance apparatus mayreceive, for example, a high definition map (HD Map) from an artificialintelligence (AI) server, and may re-determine whether the objectestimated to be present in the occupied area 802 is a potentiallythreatening object based on lane information in the HD Map. In thiscase, the vehicle collision avoidance apparatus may transform the HD Mapinto a two-dimensional occupancy grid map 803, check whether themovement of the object estimated to be present in the occupied area 802is normal based on the occupancy grid map 803 transformed from the HDMap, and determine that the object is a potentially threatening objectin response to the movement of the object not being normal as a resultof the checking. By contrast, the vehicle collision avoidance apparatusmay determine that the object is not a potentially threatening object inresponse to the movement of the object being normal as a result of thechecking. For example, based on the lane information in the occupancygrid map 803 of the HD Map, the vehicle collision avoidance apparatusmay determine that an object estimated to be present in the occupiedarea 802 is not entering a lane in which a vehicle 804 is travelling butrather is entering another lane located next to the lane in which thevehicle 804 is travelling, and thereby determine that the object is ageneral object that is unlikely to threaten the vehicle 804.

In addition, based on the occupancy grid map 803 of the HD Map, thevehicle collision avoidance apparatus may check the size of the objector the number of objects estimated to be present in an occupied area ina third occupancy grid map 805. For example, based on the occupancy gridmap 803 of the HD Map, the vehicle collision avoidance apparatus maycheck the size of a first object 807, a second object 808, and a thirdobject 809 estimated to be present in an occupied area 806 in the thirdoccupancy grid map 805, and the number of objects (e.g., three).

FIGS. 9 and 10 are diagrams illustrating an example processing methodwhen the potentially threatening object in the vehicle collisionavoidance apparatus is checked.

Referring to FIG. 9, the vehicle collision avoidance apparatus in thevehicle may transform each of the point cloud maps received from theLIDAR installed in the vehicle at set intervals from a three-dimensionalpoint cloud map to a two-dimensional occupancy grid map, and maydetermine whether a potentially threatening object that is likely tothreaten the vehicle is present within a set range with respect to thevehicle based on the difference between the plurality of occupancy gridmaps.

In response to a determination that a potentially threatening object 901is present, the vehicle collision avoidance apparatus may set an areacorresponding to spatial coordinates 902 of the potentially threateningobject as a region of interest 1001 in the image received from thecamera installed in the vehicle, and identify a type of potentiallythreatening object 901 in the set region of interest 1001.

FIG. 11 is a flowchart illustrating an example of a vehicle collisionavoidance method. Here, the vehicle collision avoidance apparatusimplementing the vehicle collision avoidance method may generate anobject identification algorithm and store the generated objectidentification algorithm in a memory. The object identificationalgorithm may be a neural network model trained to detect an object in adesignated area in a collected image and identify the type of thedetected object.

Referring to FIG. 11, in step S1101, the vehicle collision avoidanceapparatus may receive, from a LIDAR installed in a vehicle, a pointcloud map of a surrounding environment within a set range, and receive,from a camera installed in the vehicle, an image of the surroundingenvironment. In this case, the vehicle collision avoidance apparatus mayreceive the point cloud map and the image at set intervals.

In step S1102, the vehicle collision avoidance apparatus may determinewhether a potentially threatening object that is likely to threaten thevehicle is present in the point cloud map.

Here, the vehicle collision avoidance apparatus may transform each ofthe point cloud maps received from the LIDAR from a three-dimensionalpoint cloud map into a two-dimensional occupancy grid map, and as aresult of comparing each of the transformed occupancy grid maps, maycheck movement of the occupied area in which the object is estimated tobe present in each of the occupancy grid maps. The vehicle collisionavoidance apparatus may check information on the object corresponding tothe occupied area including at least one of a speed of the object, atraveling direction of the object, a size of the object, or a distancebetween the object and the vehicle, based on the movement, and determinethat the object is a potentially threatening object based on theinformation on the object.

In some examples, the vehicle collision avoidance apparatus maydetermine that the object is the potentially threatening object based onthe object being larger than a set size and moving toward the vehicle atabove a set speed, and on the distance between the object and thevehicle being shorter than a set separation distance.

In some implementations, the vehicle collision avoidance apparatus mayestimate the movement of the object based on the information on theobject, and check whether the movement of the object is normal based onlane information in an HD Map received from the server. In this case,the vehicle collision avoidance apparatus may determine that the objectis the potentially threatening object in response to the movement of theobject not being normal as a result of the checking. For example, thevehicle collision avoidance apparatus may determine that the movement ofthe object is abnormal when the object estimated to be present in theoccupied area disregards the lane information and quickly approaches theposition of the vehicle including the vehicle collision avoidanceapparatus, and thereby determine that the object is a threateningobject.

In some implementations, the vehicle collision avoidance apparatus maydetermine that the object is not a potentially threatening object inresponse to the movement of the object being normal as a result of thechecking. For example, even when an object quickly approaches thelocation of the vehicle including the vehicle collision avoidanceapparatus, in response to a prediction that the object is entering anadjacent lane based on the lane information, the vehicle collisionavoidance apparatus may determine that the movement of the object isnormal, and thereby determine that the object is a non-threateningobject that is unlikely to threaten the vehicle.

Upon the vehicle collision avoidance apparatus determining in step S1102that a potentially threatening object that is likely to threaten thevehicle is present in the point cloud map, in step S1103 the vehiclecollision avoidance apparatus may activate the avoidance travelingalgorithm to control the vehicle to perform a collision preventionoperation.

In this case, the vehicle collision avoidance apparatus may check thedistance between the potentially threatening object and the vehicle, andcause the vehicle to perform the collision prevention operation bydecelerating, accelerating, or steering the vehicle based on thedistance being longer than a set braking distance of the vehicle,thereby preparing for the possibility of collision of the vehicle withthe potentially threatening object. Further, the vehicle collisionavoidance apparatus may set a collision avoidance space for avoiding thepotentially threatening object according to the activated avoidancetraveling algorithm. In this case, the vehicle collision avoidanceapparatus may set, among the non-travelable areas of the vehicle, anavoidance area according to a set condition (for example, a crosswalk ora sidewalk where no person is present), and may set the set avoidancearea and the travelable area of the vehicle as the collision avoidancespace. That is, the vehicle collision avoidance apparatus may determinea situation in which a collision between the vehicle and the potentiallythreatening object is predicted as an emergency situation, and in theemergency situation may temporarily set as a collision avoidance spacenot only a travelable area but also an area that in a non-emergencysituation is a non-travelable area.

In step S1104, the vehicle collision avoidance apparatus may identifythe type of the potentially threatening object in a region of interestin the image corresponding to the location of the potentiallythreatening object, determine that the type of the potentiallythreatening object is a set avoidance target, and determine whether thevehicle will collide with the potentially threatening object. In thiscase, the vehicle collision avoidance apparatus may apply the objectidentification algorithm in the memory to the image of the region ofinterest, recognize the type of the potentially threatening object, anddetermine whether the recognized type of the potentially threateningobject is the avoidance target.

In addition, the vehicle collision avoidance apparatus may check thedistance between the potentially threatening object and the vehicle, anddetermine that the vehicle will collide with the potentially threateningobject based on the distance being shorter than the set braking distanceof the vehicle.

Upon the vehicle collision avoidance apparatus determining in step S1104that the type of the potentially threatening object is a set avoidancetarget and that the vehicle will collide with the potentiallythreatening object, the vehicle collision avoidance apparatus may movethe vehicle to the collision avoidance space in step S1105.

In addition, before moving the vehicle to the set collision avoidancespace, the vehicle collision avoidance apparatus may transmit a messageon the movement of the vehicle to the collision avoidance space toanother vehicle located within a set range with respect to the vehicle,so as to allow the other vehicle to recognize the movement position ofthe vehicle in advance and thereby prevent collision between the vehicleand the other vehicle.

When setting the region of interest, in response to a determination thata potentially threatening object is present in the point cloud map, thevehicle collision avoidance apparatus may set an area corresponding tothe spatial coordinates of the potentially threatening object in theimage as the region of interest. In this case, the vehicle collisionavoidance apparatus may increase a frame rate by a set multiple when thecamera photographs the region of interest so as to increase the numberof times of identifying the type of the potentially threatening objectin the received image of the region of interest, thereby increasing theaccuracy of recognizing the type of potentially threatening object abovea set reliability.

Upon the vehicle collision avoidance apparatus determining in step S1104that the type of the potentially threatening object is not the setavoidance target or that the vehicle will not collide with thepotentially threatening object, the vehicle collision avoidanceapparatus may deactivate the activated avoidance traveling algorithm,thereby reducing energy consumed by the avoidance traveling algorithm.

In addition, the vehicle collision avoidance apparatus may estimate themovement of the object based on the information on the object, anddetermine the travelable area of the vehicle based on a non-occupiedarea in which it is estimated from the occupancy grid map that no objectis present, and on the movement of the object. The vehicle collisionavoidance apparatus may adjust the determined travelable area of thevehicle based on the travelable area in the image received at the setintervals, and control the traveling of the vehicle based on theadjusted travelable area.

Implementations according to the present disclosure described above maybe implemented in the form of computer programs that may be executedthrough various components on a computer, and such computer programs maybe recorded in a computer-readable medium. Examples of thecomputer-readable medium may include, but are not limited to: magneticmedia such as hard disks, floppy disks, and magnetic tapes; opticalmedia such as CD-ROM disks and DVD-ROM disks; magneto-optical media suchas floptical disks; and hardware devices that are specially configuredto store and execute program commands, such as ROM, RAM, and flashmemory devices.

In some implementations, the computer programs may be those speciallydesigned and constructed for the purposes of the present disclosure orthey may be of the kind well known and available to those skilled in thecomputer software arts. Examples of the computer programs may includeboth machine codes produced by a compiler, and higher level languagecode that may be executed by a computer using an interpreter.

As used in the present disclosure (especially in the appended claims),the singular forms “a,” “an,” and “the” include both singular and pluralreferences, unless the context clearly states otherwise. In addition,the description of a range may include individual values falling withinthe range (unless otherwise specified), and is the same as describingthe individual values forming the range.

The above-mentioned steps constructing the method disclosed in thepresent disclosure may be performed in a proper order unless explicitlystated otherwise. The present disclosure is not necessarily limited tothe order of the steps given in the description. All examples describedherein or the terms indicative thereof (“for example,” etc.) used hereinare merely to describe the present disclosure in greater detail.Therefore, it should be understood that the scope of the presentdisclosure is not limited to the exemplary implementations describedabove or by the use of such terms unless limited by the appended claims.Also, it should be apparent to those skilled in the art that variousmodifications, combinations, and alternations can be made depending ondesign conditions and factors within the scope of the appended claims orequivalents thereof.

The present disclosure is thus not limited to the exampleimplementations described above, and rather intended to include thefollowing appended claims, and all modifications, equivalents, andalternatives falling within the spirit and scope of the followingclaims.

What is claimed is:
 1. A vehicle collision avoidance apparatus,comprising: an interface configured to: receive, from a light detectionand ranging device (LIDAR) installed at a vehicle, a point cloud maprepresenting a surrounding environment within a set range from theLIDAR, and receive, from a camera installed at the vehicle, an image ofthe surrounding environment; and a processor configured to: determinewhether an object that is expected to collide with the vehicle ispresent in the point cloud map, in response to a determination that theobject is present in the point cloud map, activate an avoidancetraveling process, set a collision avoidance space defined to avoid theobject according to the avoidance traveling process, identify a type ofthe object based on a region of interest that is set according to alocation of the object in the image, based on identifying the type ofthe object, determine whether the type of the object corresponds to aset avoidance target, and drive the vehicle to the collision avoidancespace in response to a determination that the type of the objectcorresponds to the set avoidance target.
 2. The vehicle collisionavoidance apparatus of claim 1, wherein: the point cloud map comprises aplurality of point could maps that are received from the LIDAR based onset intervals; and the processor is configured to: transform each pointcloud map from a three-dimensional point cloud map to a two-dimensionaloccupancy grid map (OGM), compare OGMs of the plurality of point couldmaps, each OGM comprising an occupied area in which one or more objectsare expected to be present, based on comparing the OGMs, determinemovement of the one or more objects in the occupied area, based ondetermining the movement of the one or more objects in the occupiedarea, determine object information corresponding to the one or moreobjects in the occupied area, the object information comprising at leastone of a speed of the one or more objects, a traveling direction of theone or more objects, a size of the one or more objects, or a distancebetween the one or more objects and the vehicle, and based on the objectinformation, determine whether the one or more objects correspond to theobject that is expected to collide with the vehicle.
 3. The vehiclecollision avoidance apparatus of claim 2, wherein the processor isconfigured to: determine that the one or more objects correspond to theobject that is expected to collide with the vehicle based on (i) thesize of the one or more objects being greater than a set size, (ii) thespeed of the one or more objects toward the vehicle being greater than aset speed, and (iii) the distance between the one or more objects andthe vehicle being less than a set separation distance.
 4. The vehiclecollision avoidance apparatus of claim 2, wherein: the interface isconfigured to receive a high definition (HD) map from a server, the HDmap comprising lane information; and the processor is configured to:estimate movement of the one or more objects based on the objectinformation, determine whether the estimated movement of the one or moreobjects is normal according to the lane information in the HD map, andbased on a determination that the estimated movement of the one or moreobjects is abnormal, determine that the one or more objects correspondto the object that is expected to collide with the vehicle.
 5. Thevehicle collision avoidance apparatus of claim 2, wherein: the interfaceis configured to receive a plurality of images from the camera at theset intervals; each OGM further comprises a non-occupied area in whichno object is expected to be present; and the processor is configured to:estimate movement of the one or more objects based on the objectinformation, based on the estimated movement of the one or more objects,determine a travelable area of the vehicle in the non-occupied area,adjust the travelable area of the vehicle based on travelable areasdetermined from the plurality of images, and control traveling of thevehicle based on the adjusted travelable area.
 6. The vehicle collisionavoidance apparatus of claim 1, wherein the processor is configured to:before setting the collision avoidance space, cause the vehicle toperform a collision prevention operation by decelerating, accelerating,or steering the vehicle based on a distance between the object and thevehicle being greater than a set braking distance of the vehicle.
 7. Thevehicle collision avoidance apparatus of claim 1, wherein the processoris configured to: determine that the vehicle is expected to collide withthe object based on a distance between the object and the vehicle beingless than a set braking distance of the vehicle; and in response to adetermination that the vehicle is expected to collide with the object,drive the vehicle to the collision avoidance space.
 8. The vehiclecollision avoidance apparatus of claim 1, wherein the processor isconfigured to: set an avoidance area comprising non-travelable areasthat the vehicle is not allowed to enter; set the avoidance area and atravelable area of the vehicle as the collision avoidance space; anddrive the vehicle to the collision avoidance space to avoid a collisionbetween the vehicle and the object.
 9. The vehicle collision avoidanceapparatus of claim 8, wherein the processor is configured to: beforedriving the vehicle to the collision avoidance space, transmit a messageabout movement of the vehicle to the collision avoidance space toanother vehicle located within a set range from the vehicle.
 10. Thevehicle collision avoidance apparatus of claim 1, wherein the processoris configured to: in response to a determination that the object ispresent in the point cloud map, set an area corresponding to spatialcoordinates of the object in the image as the region of interest;increase a frame rate of the camera for capturing images of the regionof interest; and identify the type of the object based on the imagescaptured at the frame rate.
 11. The vehicle collision avoidanceapparatus of claim 1, wherein the processor is configured to: perform anobject identification process with the image including the region ofinterest; and based on performance of the object identification process,identify the type of the object and determine whether the type of theobject corresponds to the set avoidance target, and wherein the objectidentification process comprises a neural network model trained todetect a sample object in a sample image and identify a type of thesample object.
 12. The vehicle collision avoidance apparatus of claim 1,wherein the processor is configured to deactivate the avoidancetraveling process in response to (i) a determination that the type ofthe object does not correspond to the set avoidance target or (ii) adetermination that the vehicle is not expected to collide with theobject.
 13. A vehicle collision avoidance method, comprising: receiving,from a light detection and ranging device (LIDAR) installed at avehicle, a point cloud map representing a surrounding environment withina set range from the LIDAR; receiving, from a camera installed at thevehicle, an image of the surrounding environment; determining whether anobject that is expected to collide with the vehicle is present in thepoint cloud map; in response to a determination that the object ispresent in the point cloud map, activating an avoidance travelingprocess; setting a collision avoidance space defined to avoid the objectaccording to the avoidance traveling process; identifying a type of theobject based on a region of interest that is set according to a locationof the object in the image; based on identifying the type of the object,determining whether the type of the object corresponds to a setavoidance target; and driving the vehicle to the collision avoidancespace in response to a determination that the type of the objectcorresponds to the set avoidance target.
 14. The vehicle collisionavoidance method of claim 13, wherein receiving the point cloud mapcomprises receiving a plurality of point cloud maps from the LIDAR atset intervals, and wherein the method further comprises: transformingeach point cloud map from a three-dimensional point cloud map to atwo-dimensional occupancy grid map (OGM), comparing OGMs correspondingto the plurality of point cloud maps, each OGM comprising an occupiedarea in which one or more objects are expected to be present, based oncomparing the OGMs, determining movement of the one or more objects inthe occupied area, based on determining the movement of the one or moreobjects in the occupied area, determining object informationcorresponding to the one or more objects in the occupied area, theobject information comprising at least one of a speed of the one or moreobjects, a traveling direction of the one or more objects, a size of theone or more objects, or a distance between the one or more objects andthe vehicle, and based on the object information, determining whetherthe one or more objects correspond to the object that is expected tocollide with the vehicle.
 15. The vehicle collision avoidance method ofclaim 14, wherein determining whether the one or more objects correspondto the object that is expected to collide with the vehicle comprises:determining that the one or more objects correspond to the object thatis expected to collide with the vehicle based on (i) the size of the oneor more objects being greater than a set size, (ii) the speed of the oneor more objects toward the vehicle being greater than a set speed, and(iii) the distance between the one or more objects and the vehicle beingless than a set separation distance.
 16. The vehicle collision avoidancemethod of claim 14, further comprising: receiving a high definition (HD)map from a server, the HD map comprising lane information, whereindetermining whether the one or more objects correspond to the objectthat is expected to collide with the vehicle comprises: estimatingmovement of the one or more objects based on the object information,determining whether the estimated movement of the one or more objects isnormal according to the lane information in the HD map, and based on adetermination that the estimated movement of the one or more objects isabnormal, determining that the one or more objects correspond to theobject that is expected to collide with the vehicle.
 17. The vehiclecollision avoidance method of claim 13, further comprising: beforesetting the collision avoidance space, causing the vehicle to perform acollision prevention operation by decelerating, accelerating, orsteering the vehicle based on a distance between the object and thevehicle being greater than a set braking distance of the vehicle. 18.The vehicle collision avoidance method of claim 13, wherein driving thevehicle to the collision avoidance space comprises: determining that thevehicle is expected to collide with the object based on a distancebetween the object and the vehicle being less than a set brakingdistance of the vehicle; and in response to a determination that thevehicle is expected to collide with the object, driving the vehicle tothe collision avoidance space.
 19. The vehicle collision avoidancemethod of claim 13, wherein driving the vehicle to the collisionavoidance space comprises: setting an avoidance area comprisingnon-travelable areas that the vehicle is not allowed to enter; settingthe avoidance area and a travelable area of the vehicle as the collisionavoidance space; and driving the vehicle to the collision avoidancespace to avoid a collision between the vehicle and the object.
 20. Thevehicle collision avoidance method of claim 13, wherein identifying thetype of the object in the region of interest in the image comprises: inresponse to a determination that the object is present in the pointcloud map, setting an area corresponding to spatial coordinates of theobject in the image as the region of interest; increasing a frame rateof the camera for capturing images of the region of interest; andidentifying the type of the object based on the images captured at theframe rate.